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ABSTRACT 

AXL.Net  is  a  prototype  web-based  immersive 
technology  solution  that  supports  case  method  teaching 
for  U.S.  Army  leader  development.  The  AXL.Net  system 
addresses  three  challenges:  (1)  designing  a  pedagogically- 
sound  research  prototype  for  leader  development,  (2) 
integrating  research  technologies  with  the  best  of  Web  2.0 
innovations  to  enhance  case  method  teaching,  and  (3) 
providing  an  easy  to  use  system.  Initial  evaluations  show 
that  the  prototype  application  and  framework  is  effective 
for  leader  development. 

1.  INTRODUCTION 

With  ongoing  U.S.  Army  deployments  into  complex 
environments,  there  is  an  existing  and  ever  growing  need 
to  accelerate  the  development  of  the  Army’s  junior 
leaders.  These  junior  leaders  need  to  be  developed  to 
demonstrate  effective  leadership  skills  when  facing 
situations  they  have  never  before  experienced  (Wong, 
2004)  and  as  quickly  and  efficiently  as  possible  for  the 
large  number  of  leaders  that  the  Army  needs  to  develop. 

Robert  Sternberg  and  his  colleagues  (2000)  studied 
the  practical  intelligence  of  military  leaders  -  something 
they  called  tacit  knowledge  -  and  found  that  it  fell  into 
three  broad  categories:  interpersonal  tacit  knowledge, 
intrapersonal  tacit  knowledge,  and  organizational  tacit 
knowledge.  Sternberg  and  others  found  that  leaders  gain 
expertise  and  tacit  knowledge  through  experiences  and 
reflecting  on  experiences  (Hughes,  Ginnett,  &  Curphy, 
2002;  McCall,  Lombardo,  &  Morrison,  1988;  Sternberg  et 
ah,  2000). 

Case  method  teaching  has  been  identified  as  a  way  to 
provide  sense-making  opportunities  and  is  a  widely 
recognized  technique  used  in  courses  on  leader 
development  (e.g.,  Bass,  1990;  Hays,  2005;  Hughes, 
Ginnett  &  Curphy,  2002;  Yukl,  2002).  Students  analyze  a 
realistic  case  with  the  help  of  an  instructor  or  facilitator  to 


learn  lessons  from  other  experiences  without  having  to 
live  through  the  experience  themselves. 

The  Army  Excellence  in  Leadership  (AXL)  project,  a 
collaboration  between  the  University  of  Southern 
California’s  Institute  for  Creative  Technologies  (ICT)  and 
the  U.S.  Army  Research  Institute  (ARI),  supports  tacit 
knowledge  acquisition  in  leaders,  with  focus  on 
interpersonal  tacit  knowledge.  One  result  of  the  AXL 
project  is  AXL.Net,  a  prototype  web-based  immersive 
technology  solution  that  supports  case  method  teaching 
for  U.S.  Army  leader  development. 

The  overarching  design  for  AXL.Net  was  based  on 
the  traditional  case  method  teaching  approach  (Corey, 
1999;  Gentile,  1990;  Golich  et  al.,  2000;  Hill,  Gordon  & 
Kim,  2004); 

1.  Review  learning  objectives  and  be  immersed  in  the 
case. 

2.  Familiarize  with  the  basic  story  points  -  identify 
roles/responsibilities,  story  points,  key  events. 

3.  Critically  analyze  the  case  -  consider  stakeholder 
goals,  cause/effect,  alternate  decisions. 

4.  Synthesize  thoughts  -  develop  rules  of  thumb, 
connect  to  personal  experience/real  situations. 

The  framework,  however,  required  translation  into  an 
interactive,  computer-based  medium.  Furthermore,  the 
interactive  medium  along  with  the  use  of  filmed  cases 
provided  opportunities  for  learning  experiences  that  were 
not  previously  possible.  These  experiences  needed  to  be 
invented  for  augmenting  existing  techniques  or  creating 
wholly  new  ones.  Finally,  theories  about  the  acquisition 
of  tacit  knowledge  needed  to  be  incorporated  into  the 
design  and  approach. 

2.  AXL.NET  DESIGN 

Previously,  ICT  developed  a  fictional  filmed  case. 
Power  Hungry ,  that  was  integrated  with  an  interactive 
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computer-based  technology.  Think  Like  a  Commander — 
Excellence  in  Leadership  (TLAC-XL)  (Hill,  Gordon,  & 
Kim,  2004).  Through  evaluations  by  ARI  as  well  as  ICT’s 
own  analysis,  there  were  a  number  of  lessons  learned 
from  TLAC-XL: 

Value  of  scenarios:  TLAC-XL  was  inspired  by  the 
Think  Like  a  Commander  (TLAC)  teaching  approach 
(Shadrick  &  Lussier,  2004),  but  whereas  TLAC  focused 
on  the  tactical  situation,  the  TLAC-XL  concept  expanded 
the  scenario  scope  by  considering  the  interpersonal 
aspects  of  leadership.  The  tactical  scenario  became  the 
context  for  examining  key  leadership  issues  that 
influenced  the  outcome  of  a  situation.  Interleaving 
leadership  issues  with  the  tactical  scenario  enabled 
students  to  see  how  specific  interpersonal  skills  can 
impact  an  operation.  The  Power  Hungry  case  and  TLAC- 
XL  software  were  found  to  be  memorable  and  engaging 
(Zbylut  &  Ward,  2004a),  and  the  Power  Hungry  case  has 
since  been  widely  used  in  other  instructional  settings. 

Need  for  scaffolding:  TLAC-XL  used  a  synthetic 
mentor  in  place  of  a  human  instructor  counterpart.  The 
synthetic  mentor  posed  a  series  of  highly  structured 
questions  related  to  the  eight  TLAC  topics.  The  intent  was 
to  have  the  student  think  on  their  own  and  generate 
responses  to  the  questions  rather  than  selecting  an  answer 
from  a  menu.  While  the  intent  was  appropriate — this  is 
the  expectation  in  the  case  method  approach —  the 
dialogue  was  one-way  with  the  student  generating 
analyses  but  not  receiving  feedback  from  the  mentor, 
even  as  informed  follow-up  questions. 

In  addition  to  lessons  learned  from  the  TLAC-XL 
system,  the  AXL.Net  design  was  informed  by  a  number 
of  considerations. 

Case  method  teaching  with  film:  In  case  method 
teaching,  cases  are  provided  to  students  so  that  they  can 
develop  tacit  knowledge  by  vicariously  reflecting  on  a 
situation.  The  case  method  analysis  process  includes 
evaluating  the  situation,  the  course  of  action,  and  the 
consequences.  For  the  AXL  framework,  filmed  cases 
were  recognized  as  similar  to  the  most  effective  text- 
based  cases.  Sternberg,  et  ah,  (2000)  and  Harvard 
Business  School  use  text  cases  that  are  stories  containing 
embedded  leadership  insights  into  tactical  situations. 
Filmed  narratives  are  capable  of  providing  the  same  kind 
of  stories.  Some  instructors  already  incorporate  film  into 
their  classrooms  for  teaching  concepts.  United  States 
Military  Academy  instructors  often  use  film  clips  to  serve 
as  discussion  points  within  their  Military  Leadership 
classes.  A  variety  of  instructors  also  have  adopted  the 
AXL  Power  Hungry  scenario,  along  with  the  facilitator’s 
guide  (Zbylut  &  Ward,  2004b)  that  identifies  the 
scenario’s  embedded  leadership  issues  and  provides 
suggested  discussion  questions,  for  use  in  their 


classrooms.  An  additional  film.  Tripwire ,  representing  a 
fictional  scenario  in  Iraq,  was  since  completed  in  the 
course  of  the  overall  AXL  project. 

Interactive  multimedia  exercises:  While  AXL.Net 
supports  text-based  cases,  one  of  its  strengths  is  handling 
rich  media  content.  The  technologies  developed  to 
address  this  challenge  include: 

•  The  ability  to  search  the  movie:  in  some  ways, 
similar  to  skimming  the  book  and  using  the 
index. 

•  The  ability  to  bookmark  specific  film  references 
(See  Figure  1):  similar  to  making  margin  notes 
or  highlighting  the  text. 

•  Character  interviews:  similar  to  interviewing 
“protagonists”  of  a  case  that  come  to  a  classroom 
to  answer  questions. 

Enabling  sense-making:  Student  participation  in  a 
discussion,  reflection,  and  exposure  to  different  points  of 
view  are  found  to  help  a  student’s  sense-making  process 
(Brown,  Collins  &  Duguid,  1989;  Golich,  et  ah,  2000; 
Palus,  et  ah,  2003).  In  traditional  classroom-based  case 
method  teaching,  a  case  method  instructor  typically  leads 
a  discussion  through  the  use  of  questions  (Gentile,  1990; 
Golich  et  ah,  2000).  This  activity  is  not  easily  duplicated 
in  a  distance  learning  environment  without  an  instructor. 
One  solution  in  AXL.Net  drew  on  the  instructional  design 
principles  of  guided  analysis  rather  than  pure  discovery 
learning  (Mayer,  2004).  The  system  presented  a  deliberate 
set  of  questions  authored  to  provide  opportunities  for 
reflection.  The  system  also  supported  forced  choice 
questions  (e.g.,  multiple  choice,  true/false)  that  helped 
determine  the  state  of  the  student  and  allowed  feedback  or 
branching  that  leads  to  tailored  exercises  or  open-ended 
questions.  For  example,  the  system  may  ask  whether  a 
student  has  experience  with  the  depicted  situation  and 
branch  to  a  set  of  questions  that  depend  on  this  answer. 
Another  solution  was  the  implementation  of  a  critical 
issues  tool.  The  tool  asked  the  student  to  select  and  order 
from  a  master  list  the  critical  issues  that  affected  the 
outcome  of  the  cases.  Based  on  the  students’  choices  and 
rankings,  the  system  provided  the  student  with  feedback 
about  their  choices  that  included  drawing  attention  to  the 
issues  that  they  did  not  select. 

AXL.Net  provides  different  points  of  view  through  a 
virtual  (though  currently  non-animated)  facilitator,  CPT 
Stewart ,  and  peer  responses.  The  system  and  modules 
were  also  designed  for  small  groups  of  students  to 
complete  together,  discussing  the  questions  with  each 
other,  without  an  instructor.  The  character  interview 
capability  originally  used  in  TLAC-XL  was  included  and 
enhanced.  Students  were  able  to  interview  characters  at 
any  stage  in  the  analysis  to  encourage  consideration  of  the 
“backstory”  and  the  different  motivations  of  the 
characters.  A  mediation  capability  was  integrated. 


2 


Army  Excellence  in  Leadership  -  AXL.net 


/X 


©  http://axlserver.ict.usc.edu/axl/index.php?q=7 


▼  Q 


Tripwire: 
Balancing 
Mission  and 
Troop  Safety 

Question  Progress 


Restart  Course 

This  will  hloe  your 
old  answers  and 
allow  you  to  start 
over. 


Reflect  on  key  moments  in  the  scenario.  Think  about  when  CPT  Holston  describes  the 
mission.  Bookmark  point(s)  in  the  movie  when  he  does  so,  noting  briefly  what  the 
implication  of  his  message  is. 


He  describes  his  goals  here: 

1)  keeping  the  town  "cold" 

2)  protect  his  soldiers 

3)  deal  with  arriving  official 


He  describes  them  again  here  though  slightly  differently: 

1)  protect  soldiers 

2)  support  the  meeting 

3)  keep  the  town  under  control 


Transferring  data  from  axlserver.ict.usc.edu... 


Fig.  1:  The  video  bookmarking  tool  using  a  filmed  case  in  AXL.Net. 


providing  “bridging”  dialogue  between  a  student’s 
question  and  the  available  responses  from  a  character 
(Gandhe,  Gordon,  &  Traum,  2006).  Interview  clips  were 
also  integrated  directly  into  the  instruction. 

Addressing  the  limitations  of  traditional 
approaches:  One  of  the  limitations  of  case  method 
teaching  has  been  the  differences  in  the  experience  level 
of  facilitators  (Golich  et  ah,  2000).  The  AXL.Net  system 
allows  expert  case  method  teachers  to  develop  online 
modules  that  can  then  be  accessed  and  used  by  other 
instructors.  As  previously  described,  an  “expert” 
facilitator  was  embedded  within  the  system  to  provide 
guidance  and  feedback,  and  AXL.Net  supports  branching 
and  other  tailoring  of  the  system. 

Another  limitation  of  case  method  teaching  has  been 
how  well  interpersonal  issues  can  be  depicted  in  a  paper- 
based  case.  By  presenting  the  case  with  film,  students  can 
experience  interpersonal  situations  and  conflicts  very 
similar  to  real  life:  in  real-time,  with  many  other 
distractions,  not  knowing  what  comes  next,  not  being  able 


to  easily  skim,  and  observing  body  language  and  other 
non-verbal  indicators  to  read  the  subtext.  Using  films  as 
cases  also  addresses  the  inaccessibility  of  text-based  cases 
(Kutner,  Greenburg  &  Baer,  2005),  allowing  students  to 
engage  with  other  media  besides  text. 

Leveraging  Web  2.0  innovations:  AXL.Net  takes 
advantage  of  the  web,  particularly  for  usability  and  Web 
2.0  innovations.  Core  principles  of  Web  2.0  applications 
include  dynamic  user-generated  content  (remixability), 
lightweight  data  and  service  structures,  and  simple  but 
rich  user  experiences  (O’Reilly,  2005).  By  applying  these 
core  principles  to  the  system  infrastructure  design  as  well 
as  the  user  interface  design,  the  result  was  a  system  that 
could  be  easily  modified  or  extended.  The  system  takes 
advantage  of  Asynchronous  JavaScript  and  XML  (AJAX) 
techniques  to  reduce  load  time  and  bandwidth 
requirements  for  the  media-heavy  AXL.Net.  AXL.Net 
was  also  implemented  as  a  dynamic  content  management 
system  making  authoring  and  customization  of  toolsets 
easier  for  both  instructors  and  system  developers. 
AXL.Net  applies  the  transparent  integration  of  media 
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formats,  such  as  image  file  formats  JPG,  GIF,  and  BMP, 
as  well  as  movie  file  formats  for  QuickTime  and 
Windows  Media,  that  is  possible  with  the  web.  Finally, 

and  most  critically,  AXL.Net  was  structured  to 
maximize  innovation  opportunities  for  its  users.  AXL.Net 
is  designed  for  two  different  users:  the  students  who  learn 
from  the  teaching  modules  and  the  content  creators  who 
author  modules.  AXL.Net  is  not  tied  to  specific  modules 
or  cases,  allowing  users  to  “remix”  their  own  content. 

3.  EVALUATION  METHOD 

In  the  first  half  of  2006,  ARI  conducted  initial  pilot 
tests  with  a  cultural  awareness  module  for  the  Tripwire 
case  that  used  the  AXL.Net  system  and  framework 
described  above.  Forty-four  captains  (CPTs),  six  first 
lieutenants  (lLTs),  and  five  second  lieutenants  (2LTs) 
from  three  U.  S.  Army  installations  completed  the 
module.  Of  the  officers,  12  had  deployed  twice  to  the 
Middle  East,  30  had  deployed  once  to  the  Middle  East, 
and  13  had  not  deployed  to  either  Iraq  or  Afghanistan. 

Each  officer  participated  in  an  AXL  session  typically 
consisting  of  two  to  four  individuals.  Officers  watched 
Tripwire  as  a  group  on  a  laptop  computer  and  then 
independently  completed  a  set  of  measures.  After 
completing  the  measures,  officers  began  the  interactive 
phase  of  the  cultural  awareness  module.  The  module 
consisted  of  a  series  of  open-ended  and  close-ended 
questions  to  prompt  discussion  about  the  cultural  issues  in 
Tripwire,  particularly  concerning  the  treatment  of  locals, 
managing  the  perceptions  of  locals,  and  dealing  with 
IEDs  in  an  urban  environment.  Discussion  lasted 
approximately  45  to  90  minutes.  Upon  completing  the 
module,  officers  independently  completed  another  set  of 
measures.  Descriptions  of  the  measures  are  provided  in 
subsequent  paragraphs. 

Affect.  After  watching  the  film,  officers  completed 
the  Positive  and  Negative  Affect  Schedule  (PANAS; 
Watson,  Clark,  &  Tellegen,  1988).  Officers  indicated  the 
extent  to  which  they  experienced  20  emotions  while 
watching  the  film.  The  20  items  were  anchored  on  a  five- 
point  scale  ranging  from  “very  slightly  or  not  at  all”  (1)  to 
“extremely”  (5).  Positive  affect  (a  =  .91)  was  the  mean  of 
10  positive  emotions,  such  as  active,  excited,  and  proud. 
Negative  affect  (a  =  .85)  was  the  mean  of  10  negative 
emotions,  such  as  hostile,  afraid,  and  distressed. 

Reactions  to  the  film.  After  completing  the  PANAS, 
officers  completed  15  items  that  addressed  reactions  that 
they  had  to  the  film.  These  items  were  anchored  on  a  7- 
point  scale  ranging  from  “strongly  disagree”  (1)  to 
“strongly  agree”  (7).  Six  items  assessed  how  believable 
and  complex  the  characters  in  the  film  were  (character 
depth;  a  =  .78)  and  four  items  assessed  the  degree  to 


which  officers  felt  involved  in  the  film’s  storyline  (a  = 
.79).  Survey  items  also  assessed  whether  officers  found 
the  film  confusing  and  realistic.  Additionally,  officers 
completed  two  items  that  assessed  their  preference  for 
filmed  case  studies  over  traditional  methods  of  instruction 
found  in  the  Army.  These  items  were,  “In  an  educational 
context,  I  would  rather  watch  a  film  like  this  than  read  a 
scenario,”  and  “In  an  educational  context,  I  would  prefer 
to  discuss  a  film  like  this  than  listen  to  a  PowerPoint 
briefing.” 

Reactions  to  training.  Officers  completed  16  items 
about  various  reactions  to  the  AXL  system  after  they 
finished  the  cultural  awareness  module.  Items  were 
anchored  on  a  7-point  scale  ranging  from  “strongly 
disagree”  (1)  to  “strongly  agree”  (7).  Five  items  (global 
reactions,  a  =  .80)  assessed  the  perceived  value  of  the 
AXL  tool,  ease  of  use,  how  informative  the  training  was, 
how  responsive  the  training  was  to  the  officer’s 
educational  needs,  and  how  frustrating  the  training 
approach  was  (reverse-scored).  Four  items  assessed 
whether  officers  believed  that  the  cultural  awareness 
module  made  them  think  (stimulated  thinking,  a  =  .75). 
Example  items  from  the  stimulated  thinking  scale 
included:  “The  questions  included  as  part  of  the  training 
made  me  think  critically  about  issues  in  the  scenario”  and 
“The  training  made  me  think  about  the  scenario  in  a 
different  way  than  I  usually  would.”  Officers  also 
completed  four  items  that  addressed  their  reactions  to  the 
character  interview  features  of  the  AXL  system  (character 
interactivity,  a  =  .81).  A  single  item,  “I  can  apply  some  of 
the  things  that  I  learned  here  to  my  activities  as  a  leader,” 
assessed  whether  officers  believed  they  would  transfer 
knowledge  gained  during  the  AXL  module  to  their  job 
(training  applicability).  Another  item,  “I  wanted  more 
feedback  about  how  I  was  doing  during  the  training 
modules,”  was  used  to  determine  whether  officers  wanted 
the  AXL  system  to  provide  more  feedback. 

Emphasis  on  cultural  issues.  Officers  completed  a 
rank  ordering  task  on  paper  that  consisted  of  21  actions 
that  the  fictional  characters  of  the  scenario  could  have 
performed  differently  during  Tripwire.  Eight  of  the 
actions  were  relevant  to  cultural  issues  presented  in  the 
cultural  awareness  module.  Examples  of  cultural  issues 
included,  “ CPT  Holston  should  have  refrained  from 
shooting  the  backpack  with  the  Koran,”  and  “ 1LT  Porter 
should  have  refrained  from  striking  the  second 
interpreter.”  An  example  of  a  non-cultural  issue  was 
“ CPT  Holston  should  have  ensured  that  everyone  had  a 
common  understanding  of  the  mission  and  his  intent.” 

Officers  were  told  to  select  the  top  seven  issues  in  the 
film  and  then  rank  them  according  to  importance.  Officers 
completed  the  rank-ordering  task  twice — once  after 
watching  the  film  (but  before  discussion)  and  once  after 
discussion.  This  allowed  the  researchers  to  examine  if 
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officers  deemed  the  cultural  issues  to  be  more  important 
after  participating  in  discussion  of  the  scenario. 

Behavioral  judgment.  Officers  completed  eight  items 
that  asked  them  to  choose  between  two  courses  of  action 
that  could  have  been  taken  during  Tripwire.  The  forced- 
choice  items  represented  a  heterogeneous  set  of  issues 
intentionally  embedded  in  the  case  study,  including 
maintaining  one’s  health  versus  self-sacrifice  for  the 
mission,  accomplishing  the  mission  versus  protecting 
one’s  Soldiers,  listening  to  the  advice  of  Soldiers  versus 
maintaining  command  authority,  and  neutralizing 
potential  IED  threats  versus  offending  Iraqi  civilians.  For 
each  item,  an  officer  received  one  point  for  selecting  the 
better  course  of  action  (i.e.,  consistent  with  the  teaching 
goals  built  into  Tripwire).  A  behavioral  judgment  score 
was  then  computed  by  summing  across  the  eight  items. 
The  behavioral  judgment  task  was  given  before  and  after 
discussion  of  the  scenario. 


4.  EVALUATION  RESULTS 
4.1  Trainee  Reactions 

Means  and  standard  deviations  for  the  variables  are 
presented  in  Table  1.  Results  indicated  that  reactions  to 
the  film  were  positive,  with  officers  rating  the  film  as 
involving  (M  =  5.55,  SD  =  1.03)  and  characters  as 
complex  (M  =  5.35,  SD  =  .93).  In  general,  officers  tended 
to  view  Tripwire  as  realistic,  although  substantial 
variability  existed  among  leaders  (M  =  4.96,  SD  =  1.71). 
Regardless  of  whether  the  film  was  perceived  as  realistic, 
officers  overwhelmingly  preferred  watching  a  film  to 
reading  a  scenario  (M  =  6.60,  SD  =  .81)  or  listening  to  a 
PowerPoint  presentation  ( M  =  6.76,  SD  =  .58). 

Reactions  to  the  training  package  as  a  whole  also 
tended  to  be  positive.  Officers’  global  reactions  to  the 
training  were  positive  (M  =  5.64,  SD  =  .84),  and  officers 
indicated  that  the  cultural  awareness  module  was  thought 
provoking  (M  =  5.39,  SD  =  .82).  Likewise,  officers 
indicated  they  could  apply  something  that  they  learned 
during  the  module  to  their  activities  as  a  leader  (M  =  5.56, 
SD  =  1.18).  Officers  were  somewhat  neutral  with  respect 
to  whether  they  believed  the  AXL  system  should  provide 
more  feedback  (M  =  3.45,  SD  =  1.71),  with  some 
individuals  wanting  more  feedback  than  others.  The 
character  interview  features  received  slightly  positive 
ratings  (M  =  4.53,  SD  =  1.23),  but  ratings  suggest  that 
some  of  the  character  functionality  in  the  system  needs 
improvement. 


4.2  Learning 

A  paired-samples  f-test  was  conducted  to  examine  if 
discussing  the  film  impacted  how  leaders  prioritized  the 
types  of  issues  embedded  in  the  film.  Results  indicated 
that  officers  placed  stronger  emphasis  on  cultural  issues 
after  participating  in  discussion  (M  =  3.11  issues,  SD  = 
1.39)  than  they  did  before  discussion  (M  =  2.64  issues,  SD 
=  1.17),  1(35)  =  2.50,  p  <  .05.  These  findings  provide 
evidence  that  the  cultural  awareness  module  resulted  in  a 
conceptual  shift  in  leaders’  understanding  of  the  cultural 
elements  embedded  in  Tripwire,  such  that  officers  viewed 
cultural  issues  as  significantly  more  important  after 
having  completing  the  AXL  module. 

A  paired-samples  f-test  also  was  used  to  examine 
whether  discussing  the  film  resulted  in  better  judgment  by 
leaders.  Results  indicated  that  leaders  tended  to  exhibit 
better  judgment  about  behavioral  courses  of  action  after 
discussion  (M  =  5.08,  SD  =  1.30)  than  they  did  before 
discussion  ( M  =  4.82,  SD  =  1.22),  f(54)  =  2.08,  p  <  .05. 
Such  findings  suggest  that  completing  the  cultural 
awareness  module  impacted  leaders’  judgment  in  a  way 
consistent  with  the  teaching  goals  of  Tripwire. 


Table  1 

Descriptive  Statistics  for  Study  Variables  (N  =  55) 


M 

SD 

Pre-Discussion  Measures 

Positive  Affect 

3.03 

.85 

Negative  Affect 

1.82 

.67 

Character  Depth 

5.35 

.93 

Involving  Film 

5.55 

1.03 

Confused 

1.78 

.92 

Realism 

4.96 

1.71 

Preference  for  Film  over  Text  Scenario 

6.60 

.81 

Preference  for  Film  over  PowerPoint 

6.76 

.58 

Emphasized  Cultural  Issues  (Tl) a 

2.64 

1.17 

Behavioral  Judgment  Pretest 

4.82 

1.22 

Post-Discussion  Measures 

Global  Training  Reactions 

5.64 

.84 

Stimulated  Thought 

5.39 

.82 

Character  Interactivity 

4.53 

1.23 

Training  Applicability 

5.56 

1.18 

Wanted  Feedback 

3.45 

1.71 

Emphasized  Cultural  Issues  (T2) a 

3.11 

1.39 

Behavioral  Judgment  Posttest 

5.08 

1.30 

dN=  36;  Task  given  at  two  of  three  installations. 


5 


4.3  Relationships  between  Emotions  and  Learning 


5.  Discussion 


Emotional  measures  correlated  with  some  learning¬ 
relevant  measures  in  this  study  (see  Table  2).  While 
negative  affect  was  unrelated  to  behavioral  judgment 
scores,  positive  affect  was  positively  correlated  with  the 
behavioral  judgment  posttest  score  (r  =  .33,  p  <  .05). 
Positive  affect  was  not,  however,  correlated  with  the 
behavioral  judgment  pretest  score  (r  =  .10,  p  =  ns), 
suggesting  that  positive  affect  experienced  during  the  film 
may  be  related  in  some  way  to  what  an  individual  learns 
during  the  course  of  discussion.  Both  positive  affect  and 
negative  affect  were  related  to  several  learning-relevant 
reactions  toward  the  training,  specifically  global  reactions 
(positive  affect:  r  =  .43,  p  =  .001),  evaluations  of  how 
thought-provoking  the  training  was  (positive  affect:  r  = 
.31,  p  <  .05;  negative  affect:  r  =  .44,  p  =  .001),  and 
whether  leaders  believed  they  learned  anything  that  could 
be  applied  to  their  work  (positive  affect:  r  =  .43,  p  =  .001; 
negative  affect:  r  =  .31,  p  <  .05). 

The  relationship  between  affect  and  emphasis  on 
cultural  issues  was  more  difficult  to  interpret.  Neither 
positive  affect  ( r  =  .29,  p  =  ns)  nor  negative  affect  (r  = 
.22,  p  =  ns)  were  significantly  related  to  emphasis  on 
cultural  issues  after  the  discussion.  However,  positive 
affect  was  significantly  correlated  with  how  much  leaders 
emphasized  cultural  issues  prior  to  discussion  (r  =  .48,  p 
<  .01).  Thus,  leaders  who  experienced  more  positive 
affect  during  the  film  tended  to  focus  their  attention  on 
culturally  relevant  courses  of  action  before  they  engaged 
in  discussion  with  other  leaders,  but  the  relationship 
between  positive  affect  and  emphasis  on  cultural  issues 
was  weaker  after  participating  in  discussion. 


Table  2 

Correlations  between  Affect  and  Learning  (N  =  55) 


Positive 

Affect 

Negative 

Affect 

Reactions  to  Training 

Global  Training  Reactions 

43** 

.26 

Stimulated  Thought 

.31* 

44** 

Applicability 

42** 

.31* 

Wanted  Feedback 

.04 

.01 

Learning 

Behavioral  Judgment  Pretest 

.10 

.05 

Behavioral  Judgment  Posttest 

.33* 

.13 

Emphasized  Cultural  Issues  (Tl)a 

.48** 

.21 

Emphasized  Cultural  Issues  (T2)a 

.29 

.22 

Note.  *p  <  .05,  **  p  <  .01.  dN=  36. 


The  results  of  this  study  indicated  that  the  case  study 
approach  adopted  in  the  AXL  project  and  the  AXL.Net 
system  holds  promise  in  shaping  the  knowledge  and 
judgment  of  junior  Army  officers.  Specifically,  after 
completing  the  cultural  awareness  module,  leaders 
exhibited  better  judgment  about  various  behavioral 
courses  of  action  that  could  have  been  adopted  in  the 
Tripwire  scenario.  The  cultural  awareness  module  also 
appeared  to  play  a  role  in  influencing  the  importance 
leaders  placed  on  the  cultural  issues  in  the  film,  with 
leaders  placing  stronger  emphasis  on  cultural  issues  after 
having  completed  the  module.  These  findings  are 
consistent  with  research  on  tacit  knowledge  that  suggests 
individuals  can  build  tacit  knowledge  by  reflecting  on 
their  experiences  and  the  experiences  of  others 
(Cianciolo,  Anotonakis,  &  Sternberg,  2004;  Matthew, 
Cianciolo,  &  Sternberg,  2005;  Sternberg  et  ah,  2000). 

Findings  with  respect  to  reactions  to  Tripwire  also 
were  encouraging  and  consistent  with  those  found  for 
Power  Hungry  (Zbylut  &  Ward,  2004a;  Zbylut,  Mark,  & 
Ward,  2005).  Indeed,  officers  overwhelmingly  indicated 
that  they  would  prefer  to  watch  a  film  rather  than  read  a 
case  study  or  listen  to  a  PowerPoint  presentation,  both  of 
which  are  common  instructional  techniques  used  in  Army 
education.  Such  results  suggest  that  film  is  an  appropriate 
way  for  depicting  case  study  material  in  AXL.Net. 

Officers  also  responded  positively  to  the  overall  AXL 
approach.  Officers  rated  the  cultural  module  as  valuable 
and  useful.  More  importantly,  officers  indicated  that  they 
would  be  able  to  transfer  something  that  they  learned  to 
their  activities  as  a  leader.  Furthermore,  officers  reported 
that  the  module  was  thought-provoking  —  not  only 
compelling  them  to  think  about  what  makes  a  leader 
effective,  but  helping  them  to  think  about  the  Tripwire 
scenario  in  a  different  way  than  they  usually  would. 
Taken  as  a  whole,  this  study’s  results  support  some  of  the 
basic  tenets  of  case  method  teaching;  namely,  case 
method  teaching  challenges  the  assumptions  of  students 
and  stimulates  critical  thinking  (e.g.,  Crittenden, 
Crittenden,  &  Hawes,  1999;  Jennings,  1996;  Kreber, 
2001;  Richardson,  1994;  Stewart  &  Dougherty,  1993). 

Previous  research  regarding  AXL  indicated  that  the 
first  film.  Power  Hungry ,  led  to  heightened  emotional 
arousal  and  reports  of  positive  affect  (Zbylut  &  Ward, 
2004a;  Zbylut  et  ah,  2005),  and  research  from  the  present 
study  suggests  that,  like  Power  Hungry ,  Tripwire  is 
emotionally  evocative.  Results  from  the  present  study  also 
found  a  positive  relationship  between  positive  affect  and 
behavioral  judgment  posttest  scores.  Moreover,  both 
positive  affect  and  negative  affect  were  related  to  several 
self-report  measures  indicative  of  learning — specifically, 
how  much  leaders  reported  that  the  training  stimulated 
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thought  and  the  extent  to  which  leaders  could  transfer 
knowledge  to  the  work  environment.  Due  to  the  design  of 
the  study,  it  is  unclear  whether  affect  played  a  role  in 
learning  or  was  just  a  byproduct  of  the  film  medium. 
However,  a  growing  body  of  work  on  mood,  affect,  and 
arousal  support  the  notion  that  emotions  play  an  integral 
role  in  learning.  For  example,  Murray,  Harish,  Hirt,  and 
Sujan  (1990)  found  that  individuals  in  positive  moods 
demonstrated  enhanced  cognitive  flexibility  and  had 
greater  access  to  diverse  and  unusual  information  than 
individuals  in  neutral  moods.  Both  cognitive  flexibility 
(i.e.,  being  able  to  identify  similarities  and  differences 
between  stimuli)  and  access  to  novel  information  would 
likely  enhance  case  method  discussion  because  they 
increase  the  probability  that  leaders  would  (1)  bring 
unique  perspectives  to  group  discussion,  and  (2)  be  able 
to  compare,  contrast,  and  ultimately  integrate  those 
differing  perspectives,  resulting  in  new  knowledge.  Like 
positive  affect,  negative  affect  might  play  a  productive 
role  in  learning  from  case  studies  because  negative  moods 
may  trigger  a  desire  to  process  information  more  deeply 
and  to  think  more  analytically  (George,  2000;  Sinclair, 
Mark,  &  Clore,  1994).  The  results  of  the  present  study  are 
consistent  with  the  proposition  that  negative  affect  is 
related  to  thinking;  leaders  who  reported  experiencing 
negative  affect  during  the  film  also  tended  to  report  that 
the  cultural  awareness  module  was  thought  provoking. 

In  sum,  the  findings  of  this  study  are  especially 
encouraging  given  that  many  of  the  officers  participating 
had  significant  deployment  experience,  yet  still  appeared 
to  learn  something  new  from  the  AXL.Net  module. 
Moreover,  the  AXL.Net  system  delivers  an  online  format 
for  conducting  effective  case  method  instruction  without 
requiring  intervention  from  an  instructor.  Such  a  format 
reduces  demands  on  instructor  time  in  the  form  of  lesson 
preparation  and  helps  to  deliver  interesting  educational 
content  in  an  online  learning  environment.  The  AXL 
system,  however,  is  flexible  with  respect  to  allowing 
instructors  to  be  as  involved  in  instruction  as  they  wish  to 
be.  While  instructor  involvement  is  not  required  to 
administer  AXL  modules,  instructors  can  use  AXL  films 
to  conduct  their  own  class  discussions  around  whatever 
topics  they  choose  (Zbylut  &  Ward,  2004a,  2004b). 
Additionally,  the  finalized  AXL.Net  system  will  allow 
instructors  to  facilitate  their  classroom  discussions  or 
create  their  own  online  modules  around  either  of  the  two 
existing  films  or  new  text-based  or  filmed  case  studies 
that  they  upload  into  AXL.Net’ s  library. 

Conceiving  of  and  creating  pedagogically-sound 
applications  using  new  technologies  and  new  approaches 
is  a  significant  challenge,  with  methodologies  in  the 
educational  design  research  community  still  relatively 
nascent  (Collins,  Joseph  &  Bielaczyc,  2004).  The  AXL 
project  has  attempted  to  bridge  educational  theories,  a 
novel  instructional  approach  leveraging  the  creative  skills 


of  filmmakers,  and  innovative  technologies  to  create 
useful  prototype  applications  for  U.S.  Army  Soldiers.  The 
positive  results  with  AXL.Net  indicate  a  significant  and 
productive  collaboration  between  ICT  and  ARI  in 
addressing  the  design  challenge  and  offer  promise  for 
future  collaborations  in  immersive  technologies  for 
education  and  training. 
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■.  Learning  Challenge 


“Leadership  is  influencing  people  by  providing  purpose,  direction,  and 
motivation  while  operating  to  accomplish  the  mission  and  improving  the 
organization.  ”  -U.S.  Army  Field  Manual  6-22  on  Leadership 


Expert  leadership  knowledge  is  largely  tacit 

•  Acquired  through  experience 


•  On  the  job  training  (McCall,  Lombardo,  Morrison,  1 988) 

•  Use  simulators  to  ‘learn  by  doing’ 

•  Rehearsals  (e.g.,  Duffer’s  Drift) 

•  Transfer  through  stories  (Gerrig,  1993;  Klein,  1998;  Sternberg,  2000) 

•  Experience  alone  is  not  enough 

•  Tutor  effect  (Bloom,  1984) 

•  Reflection  is  key  to  learning  (Schon,  1982,1987;  McCall  et  al.,  1988;  Chi  et  al., 
2001) 
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AXLNet  Learning  Environment 


•  AXL  case  method  approach 

•  Leadership  and  cultural  issues  in  mission  context 

•  Using  filmed  stories  as  cases 

•  Computer-based  case  analysis 

•  Structured  learning  experience 

•  Guidance  and  feedback:  to  generate  discussion  and  reflection 

•  Interactive  content  and  tools  for  “close  watching” 

•  Multiple  delivery  options:  distributed  and  classroom  learning 


•  Open  and  flexible  platform 

•  Authorable:  Not  tied  to  a  single  case  or  instructional  module 


Institute  for  Creative  Technologies 


AXL.net  Online  System 


Module:  CULTURAL  AWARENESS 

Cultural  awareness  is  a  critical  skill  for  every  warfighter,  especially  in  today's  COE.  The 
challenge  is  that  there  is  no  single  set  of  rules  to  follow  in  the  cultural  awareness  arena. 

Culture  is  a  shared  set  of  traditions,  belief  systems,  and  behaviors.  Culture  is  shaped  by 
history,  religion,  ethnic  identity,  language,  and  nationality.  It  evolves  in  response  to  many 
different  pressures  and  influences.  Culture  is  carefully  learned  through  socialization  and 
must  be  navigated  with  caution,  understanding,  and  respect. 

Note  that  Arab  culture  in  particular  is  very  rich  and  complex. 
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Course:  Tripwire:  Cultural  Awareness 


1.  CPT  Holston  should  have  requested  assistance  from  Battalion  to  support  the  VIP  meeting. 

2. 1LT  Porter  should  have  told  CPT  Holston  about  his  personal  problems. 

3.  CPT  Holston  should  have  taken  action  to  let  the  town  know  that  he  would  not  tolerate  insurgent  activity. 

4.  CPT  Holston  should  have  disciplined  1LT  Porter  for  striking  Hazem  in  front  of  the  interpreter. 

5.  CPT  Holston  should  have  kept  Hazem  under  constant  surveillance. 

6.  CPT  Holston  should  have  used  a  systematic  approach  for  dealing  with  potential  lEDs. 

7. 1 LT  Porter  should  have  refrained  from  striking  the  second  interpreter. 


Security-relevant  Actions  Self-regulation  Actions  Culturally-relevant  Actions  Leadership  Actions 


Eight  of  the  21  courses  of  action  are  classified  as  culturally-relevant  issues.  The  cultural  issues  reflect  culturally 
aware  decisions  and  actions  that  could  influence  Iraqi  behavior  and  perceptions.  As  mentioned  previously,  many 
of  these  actions  also  appear  in  the  Security-related  Actions  Category  because  many  cultural  and  security  issues 
are  Intertwined  within  the  context  of  the  scenario. 

Your  answers  indicate  that  5  of  the  seven  items  that  you  ranked  as  important  belonged  to  the  cultural  category. 
Good  job!  Cultural  issues  are  extremely  important,  not  only  In  this  fictional  case  study,  but  In  real  situations  that 
Soldiers  often  encounter  in  Iraq.  While  you  should  continue  to  think  about  the  cultural  elements  of  the  Tripwire 
scenario,  you  may  want  to  take  some  time  to  reflect  on  additional  issues  embedded  In  the  film,  such  as 
leadership.  By  focusing  on  leadership  issues  such  as  taking  care  of  Soldiers,  communicating,  and  establishing 
trust,  you  can  broaden  your  understanding  of  the  situation  depicted  in  the  film. 

You  indicated  that  CPT  Holston  should  have  taken  action  to  let  the  town  know  that  he  would  not  tolerate  Insurgent 
activity.  Good  job!  This  action  is  highly  Important  and  has  both  security  and  cultural  ramifications.  It  is  important  to 
demonstrate  to  insurgents  that  they  are  dealing  with  a  competent  military  force  and  it  is  important  to  communicate 
to  the  Iraqi  civilians  that  they  can  trust  the  Americans  to  maintain  order  and  stability  within  the  town. 

You  indicated  that  CPT  Holston  should  have  kept  Hazem  under  constant  surveillance.  Approximately  39%  of 
junior  officers  completing  a  similar  ranking  task  agree  with  your  assessment  To  interact  with  the  town,  it  is 
sometimes  necessary  to  grab  an  interpreter  off  of  the  street.  As  a  matter  of  security,  however,  it  was  imperative  to 
keep  close  supervision  on  Hazem  because  he  was  not  a  trusted  source  and  he  had  not  yet  been  cleared  by 
Battalion. 

You  indicated  that  CPT  Holston  should  have  used  a  systematic  approach  for  dealing  with  potential  lEDs.  This 
action  has  both  security  and  cultural  ramifications.  From  a  security  perspective,  CPT  Holston  and  his  Soldiers 
need  a  reliable  approach  for  neutralizing  IED  threats.  It  is  important  to  keep  in  mind,  however,  that  the  Iraqi 
population  Is  watching  how  American  forces  handle  such  threats.  For  example,  when  CPT  Holston  is  willing  to 
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Course:  Tripwire:  Cultural  Awareness 

What  do  you  think  would  have  made  a  bigger  difference  in  the  outcome  of  the  scenario? 
Choose  the  answer  you  think  is  best,  keeping  in  mind  that  neither  is  "right." 

(  A.  Handling  of  IEDs 
P  B.  Handling  of  the  second  interpreter 
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Course:  Tripwire:  Cultural  Awareness 


Why  do  you  think  the  handling  of  the  interpreter  made  a  bigger  difference  in  the  outcome 
of  the  scenario? 

Because  if  they  had  done  a  better  job  of  securing  him  and 
maybe  not  dealt  with  him  so  aggressively,  they  may  have 
figured  out  his  role. 


CPT  Holston 


1SG  Morino 


Pilot  of  Tripwire 

m  Cultural  Awareness  Module 


•  Purpose: 

•  Collect  reactions  to  module  and  instructional  approach 

•  Collect  character  interview  questions  to  improve  classification  of 
character  answers 

•  Determine  if  module  impacted  how  leaders  thought  about 
situation  depicted  in  film 

•  Assess  learning 

•  Explore  if  emotional  reactions  were  related  to  learning-relevant 
outcomes 


X  Participants 

•  55  junior  officers  from  three  Army  installations 

•  44  CPTs,  6  1  LTs,  5  2LTs 

•  1 2  deployed  twice  to  Middle  East 

•  30  deployed  once  to  Middle  East 

•  1 3  not  deployed  to  either  Iraq  or  Afghanistan 


.  Procedure 


•  Small  groups  of  officers  participated  at  a  time 

•  Upon  arriving,  officers  completed  1st  set  of  measures 
(individual) 

•  Officers  watched  film  (group) 

•  Officers  completed  2nd  set  of  measures  (individual) 

•  Officers  completed  cultural  awareness  module  (group) 

•  Officers  completed  3rd  set  of  measures  (individual) 
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.  Measures 


•  Demographic  items  (pre-film) 

•  PANAS  (post-film/pre-discussion;  Watson  et  al.,  1988) 


•  Positive  affect  (10  items,  alpha  =  .91) 

•  Negative  affect  (10  items,  alpha  =  .85) 


•  Reactions  to  film  (post-film/pre-discussion) 

•  Character  depth  (6  items,  alpha  =  .78) 

•  Involvement  with  film  (4  items,  alpha  =  .79) 

•  Confused 

•  Realism 

•  Preference  for  film  over  paper  scenarios  and  PowerPoint 
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X  Measures  (cont) 

•  Reactions  to  training  (post-discussion) 

•  Global  training  reactions  (5  items,  alpha  =  .80) 

•  Stimulated  thought  (4  items,  alpha  =  .75) 

•  Character  interactivity  (4  items,  alpha  =  .81) 

•  Applicability  of  training 

•  Wanted  feedback 


•  Learning  measures  (given  pre-  and  post-discussion) 
•  Emphasis  on  cultural  issues 
)  •  Behavioral  judgment 


\  Emphasis  on  Cultural  Issues 

•  Rank-ordering  task  indicating  what  issues  student 
prioritized  as  most  important  in  the  scenario 

•  Task  consisted  of  21  actions  that  characters  could 
have  done  differently 

•  8  of  the  actions  were  relevant  to  issues  in  module 

•  Students  selected  top  7  actions  they  viewed  as  most 


important 

•  Received  a  point  for  each  cultural  issue  selected 


\  Behavioral  Judgment  lask 

•  8  forced-choice  items  that  required  a  response  that 
was  either  consistent  or  inconsistent  with  teaching 
points  embedded  in  film 

Example: 

In  general,  it  is  better  for  a  leader  to... 

a.  Put  mission  tasks  ahead  of  physical  well-being  (e.g.,  skipping 
meals,  reducing  sleep  time)  in  order  to  get  the  job  done. 

b.  Take  time  out  to  sleep,  eat,  and  hydrate,  even  if  it  takes  some 
time  away  from  important  mission-related  tasks.* 


.  Results:  Him  Reactions 


M 

SD 

Positive  Affect 

3.03 

.85 

Negative  Affect 

1.82 

.67 

Character  Depth 

5.35 

.93 

Involving  Film 

5.55 

1.03 

Confused 

1.78 

.92 

Realism 

4.96 

1.71 

Preference  for  Film  over  Text  Scenario 

6.60 

.81 

Preference  for  Film  over  PowerPoint 

6.76 

.58 
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.  Reactions  to  Training 


M 

SD 

Global  Training  Reactions 

5.64 

.84 

Stimulated  Thought 

5.39 

.82 

Character  Interactivity 

4.53 

1.23 

Training  Applicability 

5.56 

1.18 

Wanted  Feedback 

3.45 

1.71 

X  learning 

•  After  discussion,  officers  placed  greater  emphasis  on 
cultural  issues  in  the  scenario,  t( 35)  =  2.50,  p  <  .05. 

•  M  =  2.64  cultural  issues  before  discussion 

•  M  =  3.1 1  cultural  issues  after  discussion 

•  After  discussion,  officers  performed  better  on 
judgment  task,  f(54)  =  2.08,  p  <  .05. 

•  M  =  4.82  before  discussion 


Correlations  Between 
"■  Emotions  a nd  Learning 


Reactions  to  Training 

Global  Training  Reactions 
Stimulated  Thought 
Applicability 
Wanted  Feedback 


ict 


Learning 

Behavioral  Judgment  Pretest 
Behavioral  Judgment  Posttest 


Positive  Negative 

Affect  Affect 


.43** 

.26 

.31* 

.44** 

.42** 

.31* 

o 

.01 

.10 

.05 

.33* 

.13 
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X  Evaluation  Results 

•  The  online  case  method  approach  adopted  by 
AXL.Net  appears  to  have  value 

•  Positive  reactions  to  film  and  training  approach 

•  Made  cultural  issues  salient 

•  Impacted  judgment 

•  Made  officers  think 

•  Officers  reported  they  would  be  able  to  apply  something 
they  learned 


\  AXLNet  An  Open  System 


•  AXL.Net  architecture  allows  instructional  content  to 
evolve  with  Army  and  instructor  needs 


•  AXL-designed  leadership  and  cultural  awareness  modules 

•  Authoring  capability  for  instructors  to  create  and  modify 
modules 

•  Upload  capabilities  to  add  new  scenarios 

•  Toolkit  of  interactive  components 

•  Character  interviews 

•  Critical  issues  measure 

•  Video  bookmarking  (with  video  search) 

, .  •  Question  branching/tailoring 


\  AXLNet  A  Hexible  System 


•  Multiple  options  for  instructional  paradigms 

•  Distance  learning:  one  person-one  computer  interaction 

•  Small  group  discussion:  computer-facilitated 

•  Traditional  classroom:  instructor-facilitated  discussion 


•  Multiple  user  bases 


Instructors  and  training  developers 
Army  leader  development 
Distributed  learning  environments 
Other  case  method  practitioners 


12/4/2006  use 


\  Potential  Research  Directions 


•  Longitudinal  research  necessary  to  look  at  behavioral/ 
cognitive  impact  and  to  assess  the  system  as  whole 

•  Exploration  of  the  benefits  and  dynamics  of  small 
group  discussion 

•  Investigation  of  research  technologies  to  enhance 
interactive  learning  for  classroom  exercises  and/or 
distributed  environments 

•  Developing  technology  that  will  allow  rapid  transition 
of  real-world  stories  into  scenarios  for  AXL  library 
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